The present disclosure relates to an image processing apparatus, an image processing method, and a program and, in particular, to an image processing apparatus, an image processing method, and a program that compute a subject distance on the basis of captured images obtained from different points of view or perform a correction process for computing the subject distance.
In order to compute the depth information of a subject captured by a camera, that is, a distance between the subject and the camera, techniques for analyzing the positions of the same subject in captured images using a plurality of images captured from different points of view have been developed. One of such techniques is stereo matching. Note that stereo matching is described in, for example, Yuichi Ohta and Takeo Kanade, “Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1985.
In order to compute distance information on the basis of stereo matching, corresponding feature points are searched for from a plurality of images captured from different points of view. In general, such a search process is performed within a predetermined search area in advance. However, if corresponding feature points are not detected within the predetermined area, the process abnormally terminates. In order to reliably set a detectable search area for corresponding feature points, an image conversion process, such as projective transformation, is applied, for example. In general, an image correction process or a camera adjustment process for matching the disparity of left and right images captured from different points of view to the search area of stereo matching, such as projective transformation, is referred to as a “calibration process”. Note that an existing technique using a calibration process in stereo matching is described in, for example, the following documents.
The technical document “A flexible new technique for camera calibration” at http://research.microsoft.com/˜zhang describes a configuration for obtaining a projective transformation parameter used for positioning each of images by capturing the image of a chart pattern having a plurality of feature points printed thereon (the positional relationship among the feature points is predetermined) while changing the point of view and realizing calibration on the basis of the obtained parameters.
However, this technique has a problem in that it is necessary to move the chart or a camera in order to capture a plurality of images of the chart from different points of view and, therefore, it is necessary to additionally provide a step of controlling the movement.
In addition, in order to perform calibration through only one image capturing operation, Japanese Unexamined Patent Application Publication No. 2006-250889 describes a technique in which two transparent calibration charts are overlaid, the images of two or more charts having different depths are captured at the same time, and a calibration parameter is obtained.
However, in both the processes described in “Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming” and Japanese Unexamined Patent Application Publication No. 2006-250889, in order to estimate a shift angle of an epipolar line used for making the search direction of stereo matching and the inclination direction of disparity the same, it is necessary to compute the shift angle on the basis of the geometric positional relationship among two or more feature points having different depths measured with sub-pel accuracy.
An example of a process for estimating the shift angle of an epipolar line is described next with reference to FIG. 1. In order to estimate the shift angle of an epipolar line, it is necessary to detect, as shown in FIG. 1, feature points a and b having different depths from a left camera image 11 and a right camera image 12 captured by a stereo camera 10 having left and right lenses, for example. Thereafter, it is necessary to compute a shift angle θ of an epipolar line 21 on the basis of the measurement of the geometric positional relationship among the feature points with sub-pel accuracy.
Through this process, the inclination angle is estimated using the geometric positional relationship among a few corresponding points in the vertical and horizontal directions. Thus, this process has a disadvantage in that it greatly relies on the sub-pel detection accuracy of a chart pattern. In order to increase the detection accuracy, it is necessary to employ a chart pattern having a complicated pattern that allows analysis of the positions of feature points on a sub-pel basis. However, if a chart pattern having such a complicated pattern is employed, it is difficult to detect corresponding feature points from the images and, therefore, mis-detection may frequently occur.